Skip to content

test(e2e): Add auto init from JS tests for Android #4602

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 39 commits into from
Closed

test(e2e): Add auto init from JS tests for Android #4602

wants to merge 39 commits into from

Conversation

krystofwoldrich
Copy link
Contributor

#skip-changelog

Runs the sample e2e test suit with the new manual init and the original auto init from JS.

@krystofwoldrich
chore(samples): Add package scripts for native builds, dsn and testing
cc9543f
@krystofwoldrich
Merge branch 'capture-app-start-errors' into kw-simplify-local-sample…
c20ac08 
…-testing
@krystofwoldrich
fix ci
f2a14c9
@krystofwoldrich
fix android archive path
d9fec19
@krystofwoldrich
fix ios build release
6b8f314
@krystofwoldrich
fix(sample-e2e): Fix type errors missing sentry/core and afterAll
25ab23b
@krystofwoldrich
Merge branch 'kw-fix-bad-merge' into kw-simplify-local-sample-testing
20a132e
@krystofwoldrich
fix IOS_DEVICE empty check
8b2227a
@krystofwoldrich
Merge branch 'capture-app-start-errors' into kw-simplify-local-sample…
d5e8e92 
…-testing
@krystofwoldrich
fix ios archive
20e7f55
@krystofwoldrich
test(e2e): Verify captured Errors Screen transaction
56fe4b8
@krystofwoldrich
add more tests
4cfc937
@krystofwoldrich
Merge branch 'capture-app-start-errors' into kw-test-transactions
b484b07
@krystofwoldrich
revert dsn change
86a675e
@krystofwoldrich
fix lint
71fada9
@krystofwoldrich
remove jest extended from sample e2e tests
0d6f543
@krystofwoldrich
Relax test to expect any type of app start, remove native frames as m…
c5c93d2 
…issing is also valid
@krystofwoldrich
add app, device and react native context smoke tests
def6e30
@krystofwoldrich
include transaction name in the tests
3aae5bf
@krystofwoldrich
test(e2e): Add auto init from JS tests
46782a8
@krystofwoldrich
test(e2e): Add app start crash test for iOS
6b663e0
@krystofwoldrich
split ios test config to manual and auto
bf4ee0d
@krystofwoldrich
update test job name to include test type
b7bb5fb
@krystofwoldrich
add test-ios-auto sh script
d336903
@krystofwoldrich
remove jest extended
97e1b30
@krystofwoldrich
Merge branch 'kw-test-transactions' into kw-add-auto-init-tests-ios
9d4c63a
@krystofwoldrich
Merge branch 'kw-add-auto-init-tests-ios' into kw-add-app-start-crash…
a97e5b3 
…-ios
@krystofwoldrich
fix c format
0dbd3dc
@krystofwoldrich
Merge remote-tracking branch 'origin/capture-app-start-errors' into k…
b86c577 
…w-add-auto-init-tests-ios
@krystofwoldrich
fix detox config name doesn't have auto yet
a0301bc
@krystofwoldrich
Copy link
Contributor Author

I'm currently investigating why Android tests with auto init are not returning app start.

I'm getting an error message that the app was in the background on start.

Invalid app start data: app not launched in foreground.
'Sentry Logger [warn]:', '[AppStart] Failed to retrieve the app start metrics from the native layer.'

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Feb 26, 2025 •
edited
Loading

iOS (new) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 1221.94 ms 1213.84 ms -8.10 ms
Size 3.19 MiB 4.32 MiB 1.13 MiB

Baseline results on branch: capture-app-start-errors

Startup times

Revision Plain With Sentry Diff
18a5066+dirty 1244.20 ms 1251.54 ms 7.34 ms
b6d933c+dirty 1228.02 ms 1235.32 ms 7.30 ms
1a89392+dirty 1229.00 ms 1234.78 ms 5.78 ms
c8578fb+dirty 1225.49 ms 1224.29 ms -1.20 ms
208f4af+dirty 1213.08 ms 1223.82 ms 10.73 ms
454f971+dirty 1233.51 ms 1239.18 ms 5.67 ms
8d0daf6+dirty 1227.80 ms 1236.83 ms 9.03 ms
b75148e+dirty 1202.72 ms 1212.04 ms 9.32 ms
5625ce7+dirty 1219.73 ms 1223.80 ms 4.07 ms
555070f+dirty 1223.61 ms 1227.57 ms 3.96 ms

App size

Revision Plain With Sentry Diff
18a5066+dirty 3.19 MiB 4.26 MiB 1.08 MiB
b6d933c+dirty 3.19 MiB 4.26 MiB 1.08 MiB
1a89392+dirty 3.19 MiB 4.26 MiB 1.08 MiB
c8578fb+dirty 3.19 MiB 4.30 MiB 1.12 MiB
208f4af+dirty 3.19 MiB 4.25 MiB 1.06 MiB
454f971+dirty 3.19 MiB 4.31 MiB 1.12 MiB
8d0daf6+dirty 3.19 MiB 4.30 MiB 1.12 MiB
b75148e+dirty 3.19 MiB 4.25 MiB 1.07 MiB
5625ce7+dirty 3.19 MiB 4.26 MiB 1.08 MiB
555070f+dirty 3.19 MiB 4.25 MiB 1.06 MiB

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Feb 26, 2025 •
edited
Loading

Android (new) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 331.75 ms 331.85 ms 0.10 ms
Size 7.15 MiB 8.39 MiB 1.24 MiB

Baseline results on branch: capture-app-start-errors

Startup times

Revision Plain With Sentry Diff
8d0daf6+dirty 393.58 ms 394.84 ms 1.26 ms
454f971+dirty 439.04 ms 423.88 ms -15.16 ms
1a89392+dirty 425.56 ms 530.65 ms 105.09 ms
b75148e+dirty 428.91 ms 461.26 ms 32.35 ms
18a5066+dirty 370.06 ms 414.10 ms 44.04 ms
b6d933c+dirty 398.43 ms 456.62 ms 58.19 ms
5625ce7+dirty 358.15 ms 416.65 ms 58.50 ms
555070f+dirty 388.25 ms 424.44 ms 36.19 ms
c8578fb+dirty 374.17 ms 362.06 ms -12.11 ms
208f4af+dirty 346.93 ms 402.77 ms 55.84 ms

App size

Revision Plain With Sentry Diff
8d0daf6+dirty 7.15 MiB 8.38 MiB 1.23 MiB
454f971+dirty 7.15 MiB 8.38 MiB 1.23 MiB
1a89392+dirty 7.15 MiB 8.38 MiB 1.23 MiB
b75148e+dirty 7.15 MiB 8.38 MiB 1.23 MiB
18a5066+dirty 7.15 MiB 8.38 MiB 1.23 MiB
b6d933c+dirty 7.15 MiB 8.38 MiB 1.23 MiB
5625ce7+dirty 7.15 MiB 8.38 MiB 1.23 MiB
555070f+dirty 7.15 MiB 8.38 MiB 1.23 MiB
c8578fb+dirty 7.15 MiB 8.38 MiB 1.23 MiB
208f4af+dirty 7.15 MiB 8.38 MiB 1.23 MiB

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Feb 26, 2025 •
edited
Loading

iOS (legacy) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 1230.41 ms 1225.77 ms -4.64 ms
Size 2.63 MiB 3.75 MiB 1.12 MiB

Baseline results on branch: capture-app-start-errors

Startup times

Revision Plain With Sentry Diff
18a5066+dirty 1218.00 ms 1212.27 ms -5.73 ms
b6d933c+dirty 1217.71 ms 1220.58 ms 2.87 ms
1a89392+dirty 1220.12 ms 1216.69 ms -3.43 ms
c8578fb+dirty 1218.55 ms 1217.23 ms -1.32 ms
208f4af+dirty 1209.44 ms 1217.13 ms 7.70 ms
454f971+dirty 1220.55 ms 1225.82 ms 5.27 ms
8d0daf6+dirty 1230.90 ms 1233.16 ms 2.27 ms
b75148e+dirty 1221.53 ms 1220.85 ms -0.68 ms
5625ce7+dirty 1226.98 ms 1217.77 ms -9.21 ms
555070f+dirty 1213.59 ms 1217.79 ms 4.20 ms

App size

Revision Plain With Sentry Diff
18a5066+dirty 2.63 MiB 3.70 MiB 1.06 MiB
b6d933c+dirty 2.63 MiB 3.70 MiB 1.06 MiB
1a89392+dirty 2.63 MiB 3.70 MiB 1.06 MiB
c8578fb+dirty 2.63 MiB 3.74 MiB 1.11 MiB
208f4af+dirty 2.63 MiB 3.69 MiB 1.05 MiB
454f971+dirty 2.63 MiB 3.74 MiB 1.11 MiB
8d0daf6+dirty 2.63 MiB 3.74 MiB 1.11 MiB
b75148e+dirty 2.63 MiB 3.69 MiB 1.06 MiB
5625ce7+dirty 2.63 MiB 3.70 MiB 1.06 MiB
555070f+dirty 2.63 MiB 3.69 MiB 1.05 MiB

@github-actions GitHub Actions
Copy link
Contributor

Android (legacy) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 453.38 ms 458.96 ms 5.59 ms
Size 17.75 MiB 20.12 MiB 2.37 MiB

Baseline results on branch: capture-app-start-errors

Startup times

Revision Plain With Sentry Diff
b6d933c 442.52 ms 461.82 ms 19.30 ms
8d0daf6 422.36 ms 449.40 ms 27.04 ms
454f971 436.27 ms 478.76 ms 42.48 ms
b75148e 440.04 ms 421.36 ms -18.68 ms
18a5066 427.89 ms 436.47 ms 8.57 ms
555070f 438.67 ms 428.30 ms -10.37 ms
1a89392 412.59 ms 424.69 ms 12.10 ms
5625ce7 476.74 ms 488.24 ms 11.50 ms
c8578fb 397.17 ms 415.72 ms 18.55 ms
208f4af 439.23 ms 427.31 ms -11.92 ms

App size

Revision Plain With Sentry Diff
b6d933c 17.75 MiB 20.11 MiB 2.37 MiB
8d0daf6 17.75 MiB 20.11 MiB 2.36 MiB
454f971 17.75 MiB 20.11 MiB 2.37 MiB
b75148e 17.75 MiB 20.11 MiB 2.37 MiB
18a5066 17.75 MiB 20.11 MiB 2.37 MiB
555070f 17.75 MiB 20.11 MiB 2.37 MiB
1a89392 17.75 MiB 20.11 MiB 2.37 MiB
5625ce7 17.75 MiB 20.11 MiB 2.37 MiB
c8578fb 17.75 MiB 20.11 MiB 2.36 MiB
208f4af 17.75 MiB 20.11 MiB 2.37 MiB

@krystofwoldrich krystofwoldrich force-pushed the capture-app-start-errors branch 2 times, most recently from d05e789 to 2363742 Compare June 11, 2025 18:15
@krystofwoldrich
Copy link
Contributor Author

Closing this because the branch is outdated. And the sample app is using different test suit.

Maestro instead of Detox. (See #4619 for context)

@krystofwoldrich krystofwoldrich mentioned this pull request Jun 11, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lucas-zimerman lucas-zimerman Awaiting requested review from lucas-zimerman lucas-zimerman will be requested when the pull request is marked ready for review lucas-zimerman is a code owner

@antonis antonis Awaiting requested review from antonis antonis will be requested when the pull request is marked ready for review antonis is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@krystofwoldrich